Method for robust analysis of biological activity in microscopy images

ABSTRACT

A robust object segmentation method for analysis of biological activity receives an input image and performs segmentation confidence mapping using the input image to generate segmentation confidence map output. A thresholding is performed using the object segmentation confidence map to generate a high confidence object mask output. An object segmentation confidence mapping method for analysis of biological activity receives an input image and performs segmentation decision to create segmentation decision result. A difference operation is performed to generate the segmentation decision result. A confidence mapping is performed using the difference result to generate segmentation confidence. An object level robust analysis method for biological activity receives an input image and performs object segmentation using the input image to create object segmentation result. A robust object feature measurement is performed to generate robust object feature result. An FOV level robust analysis method for biological activity receives a plurality of object feature results and performs robust FOV summary feature extraction to create robust FOV summary features. A FOV regulated feature extraction is performed to generate FOV regulated features. A FOV regulated feature extraction method for biological activity receives a plurality of object feature results and performs control object selection using the plurality of object feature results to generate control objects output. A FOV regulated feature extraction is performed to generate FOV regulation features output. An object feature FOV regulation is performed using the plurality of object feature results and the FOV regulation features to generate FOV regulated object features output. A sample level robust analysis method for biological activity receives a plurality of FOV feature results and performs robust sample summary feature extraction to create robust sample summary features. A sample regulated feature extraction is performed to generate sample regulated features. An assay level robust analysis method for biological activity receives a plurality of sample feature results and performs robust assay summary feature extraction to create robust assay summary features. An assay regulated feature extraction is performed to generate assay regulated features.

TECHNICAL FIELD

This invention relates to methods for the automated or semi-automaticanalysis of biological activity in microscopy images from life sciencesapplications that can consistently achieve high detection sensitivitywith high specificity, reproducibility and accuracy.

BACKGROUND OF THE INVENTION

High content, quantitative analysis of microscopy images is anincreasingly important tool for applications in drug discovery, basicresearch and medical diagnosis. We define image based, high contentanalysis to mean the measurement of multiple image parameters per cellor subcellular compartment or objects, across multiple cells in animage, or across multiple images. This could be done automatically in ahigh volume and high throughput manner or in a research setting thatinvolves few cells or images in a semi-automatic fashion. High contentanalysis of these assays has only become practical in drug discovery andmedical diagnosis in recent years, and is currently being adopted inbasic research.

Prior to the advent of high content screening systems, prior artapproaches in cell based screening only analyzed a single averagefluorescent response of many hundreds of cells in a biological sample,usually contained in a microtiter well. A popular assay instrument thatuses this approach is the Molecular Devices FLIPR(www.moleculardevices.com). High content screening tools in drugdiscovery have been deployed since the late 1990s. These individual cellbased assays provide researchers with large amounts of biological andchemical information, and they offer important enhancements toinformation obtained through traditional high throughput screens. Highcontent assays have to date been mostly deployed to screen chemicalcompounds against biological targets (usually receptors) geneticallyover-expressed in cell culture. More recently, high content assays havebeen increasingly adopted in target discovery; an important and popularapplication is RNA interference (RNAi) assays. The same imagingequipment and image informatics can be used in either case. High contentanalysis enables the measurement of complex and biologically importantphenotypes that could not be measured in HTS, such as morphologychanges, cellular differentiation, cytoskeletal changes, cell to cellinteractions, chemotaxis and motility, and spatial distribution changeslike receptor trafficking or complex formation.

Recently, high content analysis has become vital to cell cultureautomation, which has been identified as a critical bottleneck in bothhigh content and high throughput screening. Here cell image analysiscould be adapted to measure cells in microplates, count the cells,measure the confluence of cells, and the purity of cell culture (singleor multiple clones). An example of this is a recent collaborationannounced between MAIA Scientific and The Automation Partnership (“TAPTaps MAIA Scientific's Imaging System to Enable Automated Cell Culturefor Well Plates” in Inside Bioassays Vol. 1(4) pg 1-5) to add Maia'simage analysis software to the Cello automated cell culture system.

Chemical compound screening and RNAi based protein screening areaccelerating the adoption of high content image based analysis inacademic and basic research settings. Of course, microscopy has longbeen a benchtop tool for biologists, but until recently acquiring imagesusing camera and analysis of those images has typically been low volume,low throughput, semi automatic with manual Region Of Interest (ROI)drawing and application of simple measurement tools included withstandard digital microscopy software packages such as UniversalImaging's Metamorph, NIH Image, and MediaCybernetics' ImagePro. Thisappears to be changing as the NIH makes a strong push into chemicalcompound screening for academics. The Molecular Libraries and MolecularImaging initiative(http://nihroadmap.nih.gov/molecularlibraries/index.asp) is a keycomponent of the new NIH Roadmap (Zerhouni in Science Vol. 302(3) pg.63-64 and 72, October 2003) and will offer public sector biomedicalresearchers access to small organic molecules which can be used aschemical probes to study cellular pathways in greater depth. It isintended for these assays to make use of high content and highthroughput screening approaches, and NIH funding will likely favorresearchers who adopt these types of tools. Probably a guiding case forthe MLMI initiative, the NCI funded Harvard Institute for Chemistry andCell Biology Initiative for Chemical Genetics (Stuart Schreiber: biologyfrom a chemist's perspective in DDT Vol. 9(7) April 2004, pg. 299-303)has been using high content analysis of chemical compound screens forsome time. They use chemicals in an analogous way to mutations, todissect cellular pathways and identify previously unknown pathwaycomponents.

Very recently, RNAi has been validated as a platform technology for theanalysis of protein function, and these assays benefit immensely fromhigh content analysis to interpret the phenotypic changes of a samplesubject to genetic perturbation (Carpenter, Sabatini, SYSTEMATICGENOME-WIDE SCREENS OF GENE FUNCTION, in Genetics Vol. 5 pg. 11-22,January 2004). In the near future, genome wide screens will becommonplace. Several consortia (Netherlands Cancer Institute/CancerResearch UK, Vienna's Research Institute of MolecularPathology/EMBL/Sanger Institute, Cold Spring Harbor Laboratories, andthe RNAi consortium) have announced plans to make RNAi collections forthe entire human genome. The Sloan-Kettering Institute and GE Healthcarehave recently begun a collaboration to develop a technology capable ofscanning the entire human genome in one day to analyze the function ofeach of the bodies 35,000 genes in a cellular process (seewww.amersham.co.uk/investors/IR03/rep-4.html). This gene scanningtechnology will depend heavily on high content analysis softwaredisclosed in “Harris et al. US Patent Application no. 2003/0036855Method and Apparatus for Screening Chemical Compounds”. Gene scanningwill be made available to the broad academic community via a low-endhardware and optics platform that uses the same high content analysissoftware, a trend that indicates the growing importance of analyticalsoftware relative to hardware and optics platforms that are becomingcommoditized.

There are many prior art approaches of cell analysis. “Lee, Shih-Jong J.U.S. Pat. No. 5,867,610 Method for Identifying Objects Using DataProcessing Techniques, February 1999” discloses a method for theanalysis of images of cervical Pap smear slides that enabled the firstfully automated and FDA approved Pap smear screening device. In drugdiscovery, high content screening systems utilize advanced fluorescencelight-microscopy and molecule specific fluorescent-protein tags todirectly examine the physiology of fixed and living cells. Leadingexamples of state of the art devices are disclosed in “Harris et al. USPatent Application no. 2003/0036855 Method and Apparatus for ScreeningChemical Compounds” and “Dunlay et al. U.S. Pat. No. 5,989,835 Systemfor Cell Based Screening, November 1999”.

The de facto standard for measuring assay quality in high throughput andhigh content screens is the z factor, disclosed in “Zhang et al, ASimple Statistical Parameter for Use in Evaluation and Validation ofHigh Throughput Screening Assays, in Journal of Biomolecular ScreeningVol. 4(2) pg. 67-73, 1999”. Recently, it has been proposed that the Zfactor also be used as a measure of quality for the new screens of RNAiinduced phenotypes as well (Carpenter, Sabatini, SYSTEMATIC GENOME-WIDESCREENS OF GENE FUNCTION, in Genetics Vol. 5 pg. 11-22, January 2004).It is reasonable to assume that the Z factor will see widespread use inacademia as high throughput, high content assays are adopted.

The Z factor measures the assay signal window with a dimensionlessparameter. The signal window can be thought of as the separation bandbetween the distribution of test samples and that of control samples.This window is important to reduce false positive and false negativeresults. The Z factor is defined as:$Z = {1 - \frac{\left( {{3\quad\sigma_{s}} + {3\quad\sigma_{c}}} \right)}{{\mu_{s} + \mu_{c}}}}$where σ_(s) and σ_(c) indicate the standard deviation of the sample andcontrol populations respectively, and μ_(s) and μ_(c) indicate the meanof the sample and control populations respectively. As discussed inZhang et al., the Z factor is sensitive to both data variability and thesignal dynamic range. For example, as (3σ_(s)+3σ_(c)) approaches zero(very small standard deviations), or as |μ_(s)−μ_(c)| approachesinfinity (large signal dynamic range), the Z-factor approaches 1, andthe HTS assay approaches an ideal assay. Typically, an excellent assayis one that has a Z factor score greater than 0.5.

Assay development can be thought of as an exercise in optimization ofmany assay inputs to deliver the highest possible Z factor either byincreasing signal range or reducing variation. There are many potentialsources of variation, though scientists tend to focus on biologicalvariation rather than instrument variation as that is what they candirectly control. Sources of biological variation include subtledifferences in cells resulting from cell culture variation, differencesin DNA transfection across cells, variation in imaging probe titer andprobe characteristics (such as rate of dissipation) across cells, errorsin liquid handling, and poor cell adhesion. Furthermore, high contentmeasurements can be confounded by compound related artifacts that cancause false positives and false negatives; such as fluorescentcompounds, toxic compounds and rare morphological changes that affectthe biological signal on which the assay is based.

Indeed, the evaluation of high content assay quality is fundamentallydifferent than that of HTS assay quality because the sample unit isdifferent. In HTS the sample is a single fluorescent measurementcorresponding to microtiter well. In a high content assay, the sample isa biological object upon which a measurement or set of measurements,including combined and higher order measurements, are made using highcontent image analysis. There can be hundreds of objects in a FOV, andmany FOVs per well, slide or cell array. Thus, high content analysisintroduces a new source of variation into the measurement of assayquality: image analysis.

To date there has been no discussion in the literature or marketplaceabout how robust methods can be applied to high content analysis to bothreduce measurement variation and increase the signal strength. It wouldbe greatly beneficial to the field if robust methods could be deployedthat yield a high quality assay while allowing the same or even morevariation in assay inputs. This is possible in high volume, highthroughput, microscopy image based assays because the high content imageanalysis plays a direct role in establishing both the signal dynamicrange and the population variation.

Fundamentally, high content image analysis techniques can be used toreduce measurement variation at the sample level. Current state of theart approaches have in common the production of a binary mask. A binarymask image is a 1 bit image composed of ones (foreground) and zeros(background). The binary mask image corresponds to an input image of ahigh content assay wherein image segmentation has been applied. Imagesegmentation is the association of pixels to biological objects (e.g.cells or subcellular components). In the binary mask image the whiteareas (filled with ones) correspond to objects, and the black areas(filled with zeros) corresponds to the background. Object basedmeasurements are carried out using the original input image within theregion defined by the binary masks or their surrounding regions oftensubject to adjustments such as a correction for the non-uniform responseof the imaging system across the field of view or transformation fromintensity value to optical density. Common object based measurementsinclude total intensity, average intensity, and standard deviation ofintensity within the object region. Many other morphological featuressuch as shape, texture and color measurements can also be made.

As described in “Harris et al. US Patent Application no. 2003/0036855Method and Apparatus for Screening Chemical Compounds, the basic cellmask can be used to take measurements of nuclear and cytoplasmicactivity. One example is for a two image fluorescent assay wherein oneimage corresponds to an emission filter channel that displays a Hoechstnuclear marker and a second image corresponding to a fluorescentreporter molecule describing some biological activity located in thecytoplasm. Object masks can be created by a simple threshold basedsegmentation algorithm applied to the Hoechst image, thus each objectcorresponds to the a cell nuclear region as the intensity in the Hoechstimage displays only intensity located in the cell nucleus. An erosionimage processing operation can be applied to these masks to create thenuclear mask. These masks can be used to measure the nuclear intensityin the corresponding regions of the Hoechst image. Next to measurecytoplasmic activity in the second image, a mask to represent thecytoplasm area must be created. To do this a dilation operation usingpreset parameters is applied to the original binary mask image, andareas that were one (1) in the original mask area are set to zero (0).The result is a donut shaped mask, these masks are used to measurecytoplasmic intensity in the corresponding regions of the secondfluorescent image.

A similar method is disclosed in “Dunlay et al. U.S. Pat. No. 5,989,835System for Cell Based Screening” and two examples of determining nucleartranslocation of a DNA transcription factor are discussed. Firstly, anunstimulated cell with its nucleus labeled with a blue fluorophore and atranscription factor in the cytoplasm labeled with a green fluorophore.Secondly, the nuclear binary masks are created by performing cellssegmentation on the fluorescent image corresponding to the bluefluorophore. The cytoplasm of the unstimulated cell imaged at a greenwavelength. The nuclear mask is eroded (reduced) once to define anuclear sampling region with minimal cytoplasmic distribution. Thenucleus boundary is dilated (expanded) several times to form a ring thatis 2-3 pixels wide that is used to define the cytoplasmic samplingregion for the same cell. Using the nuclear sampling region and thecytoplasmic sampling region, data on nuclear translocation can beautomatically analyzed by high content analysis on a cell by cell basis.

Binary mask based high content measurements introduce error into theassay at an early stage, in addition to instrument error such asfocusing errors and variation in illumination. Types of measurementerror are shown in FIG. 1A-4H. FIG. 1A-1D show errors in measurement onthe nuclear image. The dark regions 104, 106 are the binary masksresulting from segmentation. The true nuclear regions 100, 102 arehighlighted in checker patterns. Measurement errors result fromsegmentation errors that include over-segmentation (FIG. 1A),under-segmentation (FIG. 1B), missed segmentation (FIG. 1C) andoverlapped segmentation (FIG. 1D). As described above, the nuclear masks104, 106, 108 are used to derive cytoplasm rings 112, 114, 116 withinwhich measurements are made on the cytoplasm regions 110, 118. FIGS.1E-1H show how errors in measurements on the cytoplasm image accumulatefrom the initial segmentation errors made when creating the nuclearmasks 104, 106, 108. The cytoplasmic rings 112, 114, 116 are shown indark black overlain on the representation of the true nuclear 100, 102(checker patterns) and cytoplasm regions (dotted patterns). As disclosedabove, cytoplasmic region measurements are meant to measure thefluorescent activity of fluorophores in the cytoplasm, however types ofcommon measurement errors include measuring both the true cytoplasm andtrue background intensities within the cytoplasm ring region 112 (FIG.1E), measuring intensities corresponding to true cytoplasm, truebackground and true nuclear regions within the cytoplasm ring region 114(FIG. 1F), missing the object altogether, and the cytoplasm ring region116 measuring the cytoplasm intensity of two cells and treating it asone (FIG. 1H). This error is again accumulated and undermines derivedmeasurements such as the standard deviation of intensity, the ratio ofcytoplasmic to nuclear intensity, etc.

Similar error is accumulated in time lapse images when objects are notperfectly aligned from frame to frame. Error is introduced when thenuclear object reference mask and the true nuclear object shift overtime. As the nucleus shifts from image frame to image frame, themeasurement region corresponding to the initial binary mask increasinglyincludes background fluorescence in its measurement.

These fundamental errors in object segmentation and measurement arepropagated throughout the assay's statistics resulting in higher assayvariability and reduced signal dynamic range. Additional variation isintroduced by instrument and biological variation. It is clear then thatthere is a need for robust methods of high content analysis that allowfor a more accurate segmentation result, and more specific and sensitivemeasurements with high repeatability. These robust measurements areneeded not only at the individual object level, but also at the FOVlevel, the sample level (usually corresponding but not limited to amicrotiter plate well or slide bound tissue specimen or micro tissuearray) and the assay level.

OBJECTS AND ADVANTAGES

This invention provides a method to reduce measurement variations andimprove measurement repeatability. The robust method can be applied atdifferent levels of cellular analysis to achieve high detectionsensitivity with high specificity, reproducibility, and accuracy. Theinput to a robust analysis step at a given level does not have to be theresult of a preceding robust analysis step and the robust result of onelevel can be processed by a non-robust analysis step. The robust methodsinclude object segmentation confidence mapping, confidence basedmeasurements, features from robust estimation, FOV regulated featureextraction, sample regulated feature extraction, assay regulated featureextraction.

These novel, robust approaches will reduce assay outcome variation whileallowing the same or perhaps even more variation in assay inputs. Thiswill allow scientists to relax many of the assay input constraintsheretofore restricted to improve high content assay quality such as cellculture constraints, DNA transfection quality, limitations on probetiter and characteristics, and automation constraints. This will resultin a faster, easier and cheaper high throughput assay set up.

The primary objective of the invention is to apply robust method toreduce measurement variations and improvement measurement repeatability.A secondary objective is to provide object segmentation confidence maprather than binary segmentation masks to reduce measurement error andallows confidence based measurements. Another objective of the inventionis to allow the application of robust methods at different levels ofcellular analysis. The fourth objective of the invention is to allowobject features to be regulated by the FOV. The fifth objective of theinvention is to allow FOV features to be regulated by the sample. Thesixth objective of the invention is to allow sample features to beregulated by the assay.

SUMMARY OF THE INVENTION

A robust object segmentation method for analysis of biological activityreceives an input image and performs segmentation confidence mappingusing the input image to generate segmentation confidence map output. Athresholding is performed using the object segmentation confidence mapto generate a high confidence object mask output.

An object segmentation confidence mapping method for analysis ofbiological activity receives an input image and performs segmentationdecision to create segmentation decision result. A difference operationis performed to generate the segmentation decision result. A confidencemapping is performed using the difference result to generatesegmentation confidence.

An object level robust analysis method for biological activity receivesan input image and performs object segmentation using the input image tocreate object segmentation result. A robust object feature measurementis performed to generate robust object feature result.

An FOV level robust analysis method for biological activity receives aplurality of object feature results and performs robust FOV summaryfeature extraction to create robust FOV summary features. A FOVregulated feature extraction is performed to generate FOV regulatedfeatures.

A FOV regulated feature extraction method for biological activityreceives a plurality of object feature results and performs controlobject selection using the plurality of object feature results togenerate control objects output. A FOV regulated feature extraction isperformed to generate FOV regulation features output. An object featureFOV regulation is performed using the plurality of object featureresults and the FOV regulation features to generate FOV regulated objectfeatures output.

A sample level robust analysis method for biological activity receives aplurality of FOV feature results and performs robust sample summaryfeature extraction to create robust sample summary features. A sampleregulated feature extraction is performed to generate sample regulatedfeatures.

An assay level robust analysis method for biological activity receives aplurality of sample feature results and performs robust assay summaryfeature extraction to create robust assay summary features. An assayregulated feature extraction is performed to generate assay regulatedfeatures.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred embodiment and other aspects of the invention will becomeapparent from the following detailed description of the invention whenread in conjunction with the accompanying drawings, which are providedfor the purpose of describing embodiments of the invention and not forlimiting same, in which:

FIG. 1A shows a true nuclear region and the binary mask ofover-segmentation;

FIG. 1B shows a true nuclear region and the binary mask ofunder-segmentation;

FIG. 1C shows a true nuclear region and the binary mask of missedsegmentation;

FIG. 1D shows the binary mask overlapped segmentation;

FIG. 1E shows the true nuclear region, the true cytoplasm region and itscytoplasm ring for FIG. 1A;

FIG. 1F shows the true nuclear region, the true cytoplasm region and itscytoplasm ring for FIG. 1B;

FIG. 1G shows the true nuclear region, the true cytoplasm region and themissing cytoplasm ring for FIG. 1C;

FIG. 1G shows the true nuclear region, the true cytoplasm region and itscytoplasm ring for FIG. 1D;

FIG. 2 shows the processing flow for a cellular analysis applicationconsisting of a plurality of processing levels;

FIG. 3 shows the processing flow for the robust cellular analysis for aplurality of processing levels;

FIG. 4 shows the processing flow for an alternative robust cellularanalysis method;

FIG. 5 shows the processing flow for the object level analysis method;

FIG. 6 shows the processing flow for an alternative object levelanalysis method;

FIG. 7A illustrates a circle overlaid on a digitization pixel grid;

FIG. 7B illustrates a slightly shifted circle overlaid on thedigitization pixel grid;

FIG. 7C illustrates a segmentation mask for FIG. 7A;

FIG. 7D illustrates a segmentation mask for FIG. 7B;

FIG. 8 shows the processing flow for the robust object segmentationmethod;

FIG. 9 shows the processing flow for the object segmentation confidencemapping method;

FIG. 10 shows the processing flow for the object level robust analysismethod;

FIG. 11 shows the processing flow for the FOV level robust analysismethod;

FIG. 12 shows the processing flow for the FOV regulated featureextraction method;

FIG. 13 shows the processing flow for the sample level robust analysismethod;

FIG. 14 shows the processing flow for the sample regulated featureextraction method;

FIG. 15 shows the processing flow for the assay level robust analysismethod;

FIG. 16 shows the processing flow for assay regulated feature extractionmethod;

DETAILED DESCRIPTION OF THE INVENTION

I. Application Scenario

The robust method can be applied at different levels of cellularanalysis to achieve high detection sensitivity with high specificity,reproducibility, and accuracy. FIG. 2 shows multiple processing levelsfor a cellular analysis assay. A cellular analysis assay consists of atleast one or a plurality of processing levels 210, 212, 214, 216. Asshown in FIG. 2, at least one input image 200 is processed by an objectlevel analysis step 210. The input image 200 could contain more than onechannels such as images of different spectrum, florescence stainedimage, nucleus stained image, phase contrast image, DifferenceInterference Contrast (DIC) image, images of different focal planes, ortime lapse images containing different temporal sampling of the objectsof interest, etc. The object level analysis step 210 detects, segments,and measures the objects of interest to create the object result 202. Anobject could be a tissue section, a cell, a nucleus, a sub-cellularcomponent, or other resolvable biological object. The object result 202could contain one or more of the attributes such as object location,object mask, object feature measurements, etc. The Field Of View (FOV)level analysis step 212 processes the object result 202 from objectscontained in the same FOV to create a FOV result 204 output. A FOV oftencontains a plurality of objects. In one embodiment of the invention, aFOV corresponds to the size of an image. The FOV result could containone or more of the attributes such as FOV object counts, FOV objectfeature population statistics, FOV object relational features, etc.

The sample level analysis step 214 processes the FOV result 204 from theFOVs contained in the same sample to create a sample result output 206.A sample often contains a plurality of FOVs. In one embodiment of theinvention, a sample corresponds to a well in a well plate based highthroughput/high content screening assays. In another embodiment of theinvention, a sample corresponds to an element of a tissue array, or aslide in a slide based assay. The sample result 206 could contain one ormore of the attributes such as sample object counts, sample objectfeature population statistics, sample FOV feature population statistics,etc. The assay level analysis step 216 processes the sample results fromthe samples contained in the same assay to create an assay result output208. An assay often contains a plurality of samples. In one embodimentof the invention, an assay corresponds to a 384 well plate in a wellplate based high throughput/high content screening assays. In anotherembodiment of the invention, an assay corresponds to a set of slides ina multiple slide based assays. In yet another embodiment of theinvention, an assay corresponds to a micro tissue assay. The assayresult could contain one or more of the attributes such as assay objectcounts, assay object feature population statistics, assay sample featurepopulation statistics, etc.

The robust cellular analysis methods of this invention includerobustness enhancement for each of the plurality of processing levels toimprove the sensitivity, specificity, reproducibility, and accuracy ofcellular analysis.

As shown in FIG. 3, the input image 200 is processed by the object levelrobust analysis step 310 to create a robust object result 302. Therobust object result 302 is processed by the FOV level robust analysisstep 312 to create a robust FOV result 304. The robust FOV result 304 isprocessed by the sample level robust analysis step 314 to create therobust sample result 306. The robust sample result 306 is processed bythe assay level robust analysis step 316 to create a robust assay result308.

Note that the input to a robust analysis step at a given level does nothave to be the result of a preceding robust analysis step and the robustresult of one level can be processed by a non robust analysis step. FIG.4 shows such an example. As shown in FIG. 4, the input to the FOV levelrobust analysis 312 is the object result output 202 from the non-robustobject level analysis step 210. On the other hand, the robust FOV result304 is processed by a non-robust sample level analysis step 214. Theresulting sample result 206 is in turns processed by an assay levelrobust analysis step 316 to create robust assay result output 308.

Those skilled in the art should recognize that other coupling ofdifferent level robust and non-robust cellular analysis steps and theirvariations are anticipated and are within the scope of this invention.

II. Object Level Robust Analysis

In one embodiment of the invention, the object level analysis stepincludes two steps: an object segmentation step 504 and an objectfeature measurement step 506. The object, segmentation step 504 detectsthe object of interest region from input image(s) 200 and generates abinary mask containing the object of interest, the object segmentationmask 500. The object feature measurement step 506 uses the objectsegmentation mask 500 and the input image 200 to calculate objectfeatures of interest to be included in the object feature result output502. The processing flow of the object level analysis method is shown inFIG. 5.

In another embodiment of the invention, the object level analysis stepincludes an object classification step 600. As shown in FIG. 6, theadditional object classification step 600 inputs the object featureresult 502 and performs object classification 600 to classify the objectinto an object type 602 (predefined or automatically determined). Theobject classification step 600 could be performed automatically using apattern classification method such as the regulation decision treedisclosed in Shih-Jong J. Lee, “Regulation of Hierarchic Decisions inIntelligent Systems”, US patent application publication no.20030069652-A1, Apr. 10, 2003. It could also include semi-automaticclassification involving human in the classification or classificationreview and correction. One of the object types may include artifact thatshould not be considered in the follow-on processing. Another objecttype may include control object that should be used for regulatedfeature extraction to be described later.

II.1 Robust Object Segmentation

Prior art object segmentation process creates binary object segmentationmask. The images suitable for computer processing have to be digitizedinto digital images. The digitalization process represent an image as aset of pixels (picture elements), each pixel has an intensity value. Dueto the digitalization effect, the true boundary of an object may notcoincidence with a pixel boundary. It could fall inside a pixel andcover part of the pixel region. The portion of pixel that is coveredcould change due to slight position shift.

The prior art object segmentation method that creates binary objectsegmentation mask makes an in-object/off-object decision for each pixel.This process is inherently inaccurate and is not repeatable because ofthe hard decision to be made at a pixel level. This effect could be verysignificant when an object is small since most of its pixels could beconsidered boundary pixels after digitization. The boundary pixels arethe ones that most likely to be impacted by the effect of shift.

FIG. 7A shows a circle overlaid on a digitization pixel grid. Asegmentation method is likely to create a segmentation mask as shown inFIG. 7C. FIG. 7B shows a slightly shifted same size circle that isoverlaid on the digitization pixel grid. A segmentation method is likelyto create a segmentation mask as shown in FIG. 7D. As can be appreciatedfrom the illustration, the segmentation masks of FIG. 7C and FIG. 7D aresignificantly different even though the difference between the twocircles are just a slight shift. The object feature measurements basedon the segmentation mask could be quite different. For example, the sizeof circle will be 7 pixels based on the segmentation mask of FIG. 7C. Itwill be 8 pixels based on the segmentation mask of FIG. 7D. So thedifference is over 10%.

The robust object segmentation method of the invention creates asegmentation confidence map rather than a binary mask for each object.The segmentation confidence map 800 can be thresholded to create a highconfidence mask 802 and a low confidence mask 804. The high confidencemask 802 includes pixels that are most likely contained in the object.The low confidence mask 804 contains all pixels that could contain partof the object. The high confidence object mask 802 is good fordisplaying the segmentation results or to be used as the binarysegmentation mask for object feature measurement. The low confidenceobject mask along with the segmentation confidence map support robustobject feature measurements.

The processing flow for the robust object segmentation method is shownin FIG. 8. As shown in FIG. 8, the segmentation confidence mapping step806 processes the input image 200 to generate a segmentation confidencemap 800. The segmentation confidence map 800 is processed by athresholding step 808 to generate a high confidence mask 802 and/or alow confidence mask 804.

In one embodiment of the invention, the processing flow of the objectsegmentation confidence mapping method is shown in FIG. 9. As shown inFIG. 9, the input image 200 is processed by a segmentation decision step908 to generate a segmentation decision result output 900. A differencestep 910 processes the segmentation decision result 900 and a threshold906 to generate a difference result output 902. A confidence mappingstep 912 processes the difference result 902 and generates asegmentation confidence output 904.

In the most basic format, all object segmentation algorithms involve asegmentation decision function with a threshold for each pixel asfollows:Seg(x,y)=d{F[I,(x,y)]−Thr(x,y)}Where

-   (x,y) is a pixel position-   Seg(x,y) is the binary segmentation mask result for pixel position    (x,y);-   d{a} is an indicator function as follows:    -   d {a}=1 if a>0    -   d {a}=0 otherwise

Where I is the input image; F [I, (x,y)] is the segmentation decisionfunction for pixel (x,y) given I. F is different for differentsegmentation algorithms. Thr(x,y) is the threshold value for pixel(x,y).

A simple segmentation algorithm applying a fixed threshold value T onthe image intensity can be expressed in the above formula by setting F[I, (x,y)]=I(x,y) and Thr(x,y)=T. In this case, the segmentationdecision function simply outputs image intensity of the correspondingpixel (x,y).

A more sophisticated object segmentation method disclosed in “Lee,Shih-Jong, U.S. Pat. No. 5,867,610, Method for identifying objects usingdata processing techniques” requires the segmentation decision function,F[I, (x,y)], to be a nonlinear function and Thr(x,y) to be a function ofthe image pixel location stored as threshold images. Another objectsegmentation method disclosed in “Lee, Shih-Jong, Oh, Seho, U.S. patentapplication Ser. No. 10/410,063, Learnable Object Segmentation”, whichis incorporated in its entirety herein, uses object regions of interestsegmentation recipe to guide the object segmentation. In this case, bothF and Thr functions are defined by the recipe. F could be a pixelclassification function defined by the object regions of interestsegmentation recipe and Thr could be a function of pixels as well.

The difference function compares the segmentation decision results andthe threshold. In one embodiment of the invention, the differencefunction is a simple subtractionDiff(x,y)=F[I, (x,y)]−Thr(x,y)

In another embodiment of the invention, the difference function is anabsolute difference. The difference result can be determined for anygiven object segmentation method having segmentation decision andthreshold. The confidence mapping method of this invention applies aconfidence function to the values of the difference result as follows:C(x,y)=Conf(Diff(x,y))

The result of the confidence mapping function is the segmentationconfidence. The segmentation confidence for an image of object forms theobject segmentation confidence map.

In one embodiment of the invention, the function for confidence mappingcan be determined using at least one training image where the desiredsegmentation result for each pixel is known. The segmentation accuracycan therefore be determined for different distance values using thetraining images. The empirically determined segmentation accuracyfunction can be normalized or scaled as the confidence function.

Those skilled in the art should recognize that the empiricallydetermined segmentation accuracy function can be filtered or fitted byGassian, polynomial or other functions to yield a smooth andwell-behaved confidence function.

II.2 Robust Object Feature Measurement

II.2.1 Basic Features

Object feature measurements can be considered the application ofestimators to estimate certain attributes of an object. The attributescould be physical characteristics such as size, shape, and density of acell. Statistical based estimators are often used for the estimation.This results in statistical measurements such as area, mean intensity,etc. Typical calculations for area, mean intensity, and intensityvariance can be described as follows:${\text{Area:}\quad N} = {\sum\limits_{{({x,y})} \in O}1}$${{Mean\_ intensity:}\quad\mu} = {\frac{1}{N}{\sum\limits_{{({x,y})} \in O}{I\left( {x,y} \right)}}}$${{Intensity\_ variance:}\quad\sigma^{2}} = {\frac{1}{N - 1}{\sum\limits_{{({x,y})} \in O}\left( {{I\left( {x,y} \right)} - \mu} \right)^{2}}}$

Where O is the object mask.

Those skilled in the art should recognize that other features could becalculated from the data. For example, the higher order statistics ofthe intensity distributions such as skewness (third order moment) andKurtosis (fourth order moment) etc. In general, most of the features arederived from the estimation of parameters of the different models forthe data.

Those skilled in the art should also recognize that in the case that anobject is acquired from multiple image channels. The features could bederived from multiple images. For example the segmentation mask could bederived from one image channel and the measurements from another. Somefeatures may involve the combinations of image intensity values frommultiple images.

II.2.2 Confidence Based Measurements

If an object is represented by the segmentation confidence map generatedfrom the robust object segmentation method of the invention, theconfidence based measurements can be achieved by weighting each pixel byits confidence value as follows:$\quad{{{Area}_{c}\text{:}\quad N_{c}} = {\sum\limits_{\forall{{C{({x,y})}} > 0}}{C\left( {x,y} \right)}}}$${{Mean\_ intensity}_{c}\text{:}\quad\mu_{c}} = {\frac{1}{N_{c}}{\sum\limits_{\forall{{C{({x,y})}} > 0}}{{C\left( {x,y} \right)}*{I\left( {x,y} \right)}}}}$${{Intensity\_ variance}_{c}\text{:}\quad\sigma_{c}^{2}} = {\frac{1}{N_{c} - 1}{\sum\limits_{\forall{{C{({x,y})}} > 0}}\left( {{{C\left( {x,y} \right)}*{I\left( {x,y} \right)}} - \mu_{c}} \right)^{2}}}$

Where the pixels used including all pixels having the confidence valueC(x,y)>0.

II.2.3 Robust Measurements

The object level robust analysis includes an object segmentation step504 using input image 200 to generate object segmentation result 1000.This is followed by a robust object feature measurement step 1004 thatuses the object segmentation result 1000 and the input image 200 togenerate robust object feature result 1002. The processing flow is shownin FIG. 10. The object segmentation method includes robust objectsegmentation method. In this case, confidence based features could bemeasured by the robust object feature measurement.

In the general statistical framework, numbers derived from datarepresent samples of a random variable. The probability distribution ofthe random variable determines the chance of samples having particularvalues. We don't know the probability distribution of the randomvariable, but by sampling it (i.e. by collecting data and makingmeasurements) we try to estimate properties of the random variable anduse them for decision making or test of hypotheses. For example, thedensity (or image intensity) of a cell is a random variable. Intensitydata of the cell are samples of the random variable. The average of agroup of data (pixel intensities) is not the mean of the randomvariable; it is (just) an estimate of the true, but unknown, mean of thepopulation.

Most of the basic features such as the average measurement are a goodestimate of the true mean under least-squares estimation. Thisestimation assumes that the noise corrupting the data is of zero mean,which yields an unbiased parameter estimate. Least-squares estimatorsimplicitly assume that the entire set of data can be interpreted by onlyone parameter vector of a given model. Numerous studies have beenconducted, which clearly show that least-squares estimators arevulnerable to the violation of these assumptions. Sometimes even whenthe data contains only one bad datum, least-squares estimates may becompletely perturbed.

The most commonly estimated quantities for a data set are the centraltendency and the dispersion of the data.

A. Central Tendency of the Data

Central tendency of the data estimates “around what value is the datacentered?” For a random variable with a normal distribution, the bestestimate of the underlying mean μ is the average of the data values. Forexample, the mean_intensity defined in the basic feature set is anaverage estimate that will yield good result when the random variable isa normal distribution.

Real signals from real data seem to have more outliers than predicted bya Gaussian distribution. The mean of the data will be distorted by anoutlier and is not necessarily the best estimate for non-normal data.Robust methods are less sensitive to outliers than are parametricmethods.

In one embodiment of the invention, a robust estimation of the intensitycentral tendency is the median value of the intensity data, when theyare ranked. That is:Median_intensity: m ₀=Median{I(x,y)|∀(x,y)εO}

The confidence based median intensity could also be calculated:Median_confidence_intensity: m _(0c)=Median{C(x,y)*I(x,y)|∀C(x,y)εO}

In another embodiment of the invention, trimmed means are used forrobust feature for central tendency of the data. Trim means calculatethe averages of the distribution after certain percentages (for example,1%, 5%, 10%, etc.) of data have been trimmed from the tails of thedistribution. Such means are robust to outliers.

In yet another embodiment of the invention, L-estimates, which arelinear combinations of order statistics are used. One “typical”L-estimate is the Tukey's trim mean, defined as the weighted average ofthe first, second, and third quartile points in a distribution, withweights ¼, ½, and ¼, respectively.

B. Dispersion of the Data

Dispersion of the data estimates “how much does the data spread aroundits central value?” for a random variable with a Gaussian distribution,a best estimate of the true variance is the estimated variance, σ². Forexample, the intensity_variance defined in the basic feature set is avariance average estimate that will yield good result when the randomvariable is a Gaussian distribution.

In one embodiment of the invention, a robust estimate of the datadispersion is the absolute deviation, which is a more robust estimate ofthe spread for non-Gaussian data. The intensity mean absolute deviationcan be calculated as follows:${{Intensity\_ Mean}{\_ AbsDev}\text{:}\quad{d}} = {\frac{1}{N}{\sum\limits_{{({x,y})} \in O}{{{I\left( {x,y} \right)} - m_{0}}}}}$

The confidence based intensity absolute deviation could also becalculated:${{Intensity\_ Mean}{{\_ AbsDev}_{c}:{d}_{c}}} = {\frac{1}{N_{c}}{\sum\limits_{\forall{{C{({x,y})}} > 0}}{{{{C\left( {x,y} \right)}*{I\left( {x,y} \right)}} - m_{0c}}}}}$

In another embodiment of the invention, interquartile range: thedifference between the 75% ile and the 25% ile values used for robustfeature for central tendency of the data. Those skilled in the artshould recognize that other robust estimates of the spread such asstandard errors and confidence intervals can be used as robust features.They are relatively robust to violations of normality and variancehomogeneity.

C. General Features

For other features that are derived from the estimation of parameters ofmodels for the data. The estimation techniques, which is insensitive tosmall departures from the idealized assumptions such as M-estimateswhich follow from maximum likelihood considerations can be used in oneembodiment of the invention. (NUMERICAL RECIPES IN C: THE ART OFSCIENTIFIC COMPUTING (ISBN 0-521-43108-5) Cambridge University Press1992, PP. 699-706).

III. FOV Level Robust Analysis

The FOV level analysis inputs a plurality of the object feature resultsfrom the objects belonging to the same FOV. The prior art FOV levelanalysis method extracts FOV summary features through simple populationstatistics from the object features. The FOV level robust analysismethod performs robust FOV summary feature extraction 1110 using theplurarity of object feature results 1100, 1102 as shown in FIG. 11. Thisresults in robust FOV summary features 1104. In addition, FOV relationalfeatures 1106 could be extracted using the plurarity of object featureresults 1100, 1102 by an object relational feature extraction stage 1112that results in FOV relational features 1106. Furthermore, a FOVregulated feature extraction stage 1114 could be applied to extract FOVregulated features 1108 from the plurarity of object feature results1100, 1102. The processing flow of the FOV level robust analysis methodis shown in FIG. 11.

III.1 Robust FOV Summary Feature Extraction

The basic FOV summary features are simple population statistics from theobject features. Example features include object counts for each objecttype that can be calculated as follows:${{Object\_ count}(t)\text{:}\quad N_{t}} = {\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = t}1}$

Where t is the object type. T(o) is the classification type of object o.FOV is the FOV of interest.

Other example features are the FOV summary statistics of objectfeatures. The object features are described in section II.2 such asarea, center and dispersion related features, etc. They consist of basicfeatures and robust features. In one embodiment of the invention, theFOV summary statistics include central tendency and dispersionstatistics. These include both non-robust and robust statistics.

In one embodiment of the invention, the FOV summary features for anobject feature F include the basic central tendency feature (mean) andbasic spread feature (variance) of FOV data such as:${{FOV\_ Mean}{\_ F}\text{:}\quad{\mu_{FOV}^{F}(t)}} = {\frac{1}{N_{t}}{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = t}{F(o)}}}$FOV_Variance_F:${\sigma_{FOV}^{2F}(t)} = {\frac{1}{N_{t} - 1}{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = t}\left( {{F(o)} - {\mu_{FOV}^{F}(t)}} \right)^{2}}}$

The object features F include basic features and robust features forarea, intensity center, spread, object shape, intensity contrasts,object intensity distribution statistics, texture, and other objectfeatures.

Those skilled in the art should recognize other general summarystatistics such as skewness (third order moment) and Kurtosis (fourthorder moment) etc. could be used. Also, other features derived from theestimation of parameters of different models could be used.

In another embodiment of the invention, the FOV summary features for anobject feature F include the robust central tendency features such asmedian or trim means that are the averages of the distribution aftercertain percentages (for example, 1%, 5%, 10%, etc.) of data have beentrimmed from the tails of the distribution. Furthermore, Tukey's trimmean, defined as the weighted average of the first, second, and thirdquartile points in a distribution, with weights ¼, ½, and ¼ can also beused. The trim mean at the p percentage can be defined as follows:${{FOV\_ Trim}{\_ Mean}^{p}{{\_ F}:{\mu_{FOV}^{pF}(t)}}} = {\frac{1}{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = {{{{{{t\&}{R{({F{(o)}})}}} > \frac{p}{2}}\&}{R{({F{(o)}})}}} < {1 - \frac{p}{2}}}}1}*{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = {{{{{{t\&}{R{({F{(o)}})}}} > \frac{p}{2}}\&}{R{({F{(o)}})}}} < {1 - \frac{p}{2}}}}{F(o)}}}$

Where R(F(o)) is the rank percentage of the feature F for object o.

The FOV summary features for an object feature F also include the robustdispersion features such as the mean absolute deviation, interquartilerange and standard errors.

The current invention includes generalized trimming. The traditionaltrim means calculate the averages of the distribution after certainpercentages (for example, 1%, 5%, 10%, etc.) of data have been trimmedfrom the tails of the distribution of the feature of interest. Thegeneralized trimming calculates the averages of the distribution aftercertain percentages of data that meet generalized trimming criteria havebeen trimmed. The generalized trimming criteria allows the trimmingconditions be derived from not only the feature of interest but alsofrom the distributions of other features. For example, the intensitytrim mean could condition on trimming the objects whose areas (shapes)are in the tails of the distribution even though intensity is thefeature of interest. The trimming criteria could also include combinedconditins so that both the objects within the tails of area (or shape)and tails of intensity distributions are excluded from the intensitytrim mean calculations. The generalized trimming allows the exclusion ofextreaneous objects (artifacts) from measurement based on not just purestatistical tail exclusion of the feature of interest. For example,overlapped nuclei have large size (even though the intensity may notfall into the tail of intensity distribution) and therefore are excludedfrom intensity measurement.

The generalized trim mean for feature F using both the p percentage of Ffeature and q percentage of Q featture can be defined as follows:$\frac{{{FOV\_ GTrim}{\_ Mean}^{pq}{{\_ F}:\quad{\mu_{FOV}^{pqF}(t)}}} = 1}{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = {{{{{{{{{{{{t\&}{R{({F{(o)}})}}} > \frac{p}{2}}\&}{R{({F{(o)}})}}} < {1 - \frac{p}{2}}}\&}{R{({Q{(o)}})}}} > \frac{q}{2}}\&}{R{({Q{(o)}})}}} < {1 - \frac{q}{1}}}}1}*{\sum\limits_{{{{o \in {FOV}}\&}{T{(o)}}} = {{{{{{{{{{{{t\&}{R{({F{(o)}})}}} > \frac{p}{2}}\&}{R{({F{(o)}})}}} < {1 - \frac{p}{2}}}\&}{R{({Q{(o)}})}}} > \frac{q}{2}}\&}{R{({Q{(o)}})}}} < {1 - \frac{q}{1}}}}{F(o)}}$III.2 Object Relational Feature Extraction

The object relational features characterize relations of multiple setsof objects by comprehensive collections of relational features. Acombination of these features could characterize subtle physical,structural or geometrical conditions under the practical arrangements ofthe object sets and sensing conditions. In one embodiment of theinvention, object relational features are the spatial mapping featureset disclosed in U.S. patent application Ser. No. 10/411,437, filed Apr.9, 2003 and entitled “Intelligent Spatial Reasoning” by Lee, Shih-Jongand Oh, Seho, which is incorporated in its entirety herein.

The spatial mapping feature set includes mean, standard deviation,skewness and kurtosis of the data. The robust statistics could beapplied to these feature set to improve the robustness of the objectrelational features.

For example, the inner distance mean feature could be made robust byusing median or trim means instead of simple average. As anotherexample, the inner distance standard deviation feature could be maderobust by using mean absolute deviation, interquartile range or standarderrors instead of simple standard deviation calculation.

Other Object Relational Features Include

-   -   Number of type 1 objects within a distance from a type 2 object    -   Number of type 1 objects within K nearest neighbors from a type        2 object    -   Mean (median, trim mean, etc.) area of type 1 objects within a        distance from a type 2 object    -   Mean (median, trim mean, etc.) area of type 1 objects within K        nearest neighbors from a type 2 object    -   Mean (median, trim mean, etc.) intensity of type 1 objects        within a distance from a type 2 object    -   Mean (median, trim mean, etc.) intensity of type 1 objects        within K nearest neighbors from a type 2 object    -   Mean (median, trim mean, etc.) contrast of type 1 objects within        a distance from a type 2 object    -   Mean (median, trim mean, etc.) contrast of type 1 objects within        K nearest neighbors from a type 2 object    -   Mean (median, trim mean, etc.) texture of type 1 objects within        a distance from a type 2 object    -   Mean (median, trim mean, etc.) texture of type 1 objects within        K nearest neighbors from a type 2 object

The above features correspond to each of the type 2 object. Therefore,the robust FOV summary feature as described in section III.1 could beapplied to the object relational features to generate the FOV summaryobject relational features.

III.3 FOV Regulated Feature Extraction

The processing flow of the FOV regulated feature extraction method isshown in FIG. 12. The FOV regulated feature extraction method inputs theobject feature results 1100, 1102 from the FOV and performs controlobject selection step 1208 that selects the control objects 1200 fromthe FOV object features. The control objects 1200 are used by a FOVregulation feature extraction step 1214 to extract FOV regulationfeatures 1202. The FOV regulation features 1202 are used by an objectfeature FOV regulation step 1210 to generate FOV regulated objectfeatures 1204. The FOV regulated object features 1204 are used by aregulated FOV summary feature extraction step 1212 to generate regulatedFOV summary features 1206.

III.3.1 Control Object Selection

In one embodiment of the invention, the control objects could be thespecially prepared standard cells. The control objects are selectedbased on the results of the object classification as shown in FIG. 6. Inanother embodiment of the invention, the control objects are thereference objects extracted from the object population. In the case, thecontrol objects are selected based on the FOV object featuredistribution. For example, the control objects could be the objectshaving the area and mean intensity within the middle 50% of thedistribution within the FOV.

III.3.2 FOV Regulation Feature Extraction

FOV regulation features can be calculated from the object featureresults of the control objects for the FOV. In one embodiment of theinvention, the FOV summary features are extracted for the FOV regulationfeatures. The FOV summary features that are suitable for the FOVregulation features include center (mean, median, trim mean, generalizedtrim mean, etc.) and dispersion (variance, mean absolute deviation,range, etc.) for features such as area, intensity, density (logintensity), integrated density, contrast, texture, etc.

III.3.3 Object Feature FOV Regulation

The object feature FOV regulation step regulates the extracted objectfeatures to create FOV regulated object features for each of the objectsbeing considered. It inputs an object feature and FOV regulationfeatures and applies FOV regulation formula to the object feature. Thisresults in FOV regulated object feature. In one embodiment of theinvention, the FOV regulation feature extraction calculates the formulais as follows:$F_{FOV\_ r} = \frac{F - {\theta\quad R_{FOV}^{1}}}{\gamma + {\left( {1 - \gamma} \right)R_{FOV}^{2}}}$Where F is the input object feature; θ is a normalization factor; R¹_(FOV) is the first FOV regulation feature such as the center feature; γis a weighting factor between 0 and 1; and R² _(FOV) is the second FOVregulation feature such as the dispersion feature.

When γ=1, the FOV regulation includes only the offset of the feature bythe first FOV regulation feature. When θ=0 and γ<1, the FOV regulationincludes only the gain adjustment of the feature by the second FOVregulation feature. When θ≠0 and γ<1, the FOV regulation includes boththe offset by the first FOV regulation feature and gain adjustment bythe second FOV regulation feature.

The object feature FOV regulation allows the removal of the FOV specificbias or background noise and variations. The removal of FOV specificvariations would enhance the repeatability and robustness of the FOVlevel analysis.

III.3.4 Regulated FOV Summary Feature Extraction

The regulated FOV summary feature extraction inputs the FOV regulatedobject features from a plurality of the objects and generates theregulated FOV summary features. The same procedure as the robust FOVsummary feature extraction as described in section III.1 could beapplied to the FOV regulated object features to generate the regulatedFOV summary features.

IV. Sample Level Robust Analysis

The sample level analysis step processes the FOV results from the FOVscontained in the same sample to create a sample result output. A sampleoften contains a plurality of FOVs. In one embodiment of the invention,a sample corresponds to a well in a well plate based highthroughput/high content screening assays. In another embodiment of theinvention, a sample corresponds to a slide in a slide based assays. Thesample result could contain one or more of the attributes such as sampleobject counts, sample object feature population statistics, sampleobject relational features, sample FOV feature population statistics,etc.

The sample level analysis inputs a plurality of the FOV feature results1300, 1302 from the FOVs belonging to the same sample. The prior artsample level analysis method extracts sample summary features throughsimple population statistics from the FOV features. The sample levelrobust analysis method performs robust sample summary feature extraction1308 using the plurarity of FOV feature results as shown in FIG. 13.This results in robust sample summary features 1304. In addition, asample regulated feature extraction stage 1310 could be applied toextract sample regulated features 1306 from the plurarity of FOV featureresults 1300, 1302. The processing flow of the sample level robustanalysis method is shown in FIG. 13.

IV.1 Robust Sample Summary Feature Extraction

The basic sample summary features are simple population statistics fromthe FOV features. Example features include object counts for each objecttype that can be calculated as follows:${{Object\_ count}^{s}(t)\text{:}\quad N_{t}^{s}} = {\sum\limits_{i \in s}{{Object\_ count}\left( {i,t} \right)}}$Where t is the object type and Object_count(i,t) is the type t objectcount of FOV i belonging to sample s.

Other example features are the sample summary statistics of FOVfeatures. The FOV features are described in section III such as FOVsummary features, object relational features, and FOV regulatedfeatures, etc. They consist of basic features and robust features. Inone embodiment of the invention, the sample summary statistics includecentral tendency and dispersion statistics. These include bothnon-robust and robust statistics.

In one embodiment of the invention, the sample summary features for aFOV feature F include the basic central tendency feature (mean) andbasic spread feature (variance) of the sample data such as:$\begin{matrix}{{{Sample\_ Mean}{\_ F}\text{:}\quad{\mu_{Sample}^{F}(t)}} = {\frac{1}{N_{t}^{s}}\quad{\sum\limits_{i \in s}{F\left( {i,t} \right)}}}} \\{{{Sample\_ Variance}{\_ F}\text{:}\quad{\sigma_{Sample}^{2\quad F}(t)}} = {\frac{1}{N_{t}^{s} - 1}{\sum\limits_{i \in s}\left( {{F\left( {i,t} \right)} - {\mu_{Sample}^{F}(t)}} \right)^{2}}}}\end{matrix}$

Those skilled in the art should recognize other summary statistics suchas skewness (third order moment) and Kurtosis (fourth order moment) etc.could be included. Also, other features derived from the estimation ofparameters of different models could be used.

In another embodiment of the invention, the sample summary features fora FOV feature F include the robust central tendency features such asmedian or trim means that are the averages of the distribution aftercertain percentages (for example, 1%, 5%, 10%, etc.) of data have beentrimmed from the tails of the distribution. Furthermore, Tukey's trimmean, defined as the weighted average of the first, second, and thirdquartile points in a distribution, with weights ¼, ½, and ¼ can also beused. The trim mean at the p percentage can be defined as follows:${{Sample\_ Trim}{\_ Mean}^{p}{\_ F}\text{:}\quad{\mu_{Sample}^{p\quad F}(t)}} = {\frac{1}{\sum\limits_{{{{{{{i \in s}\&}\quad{R{({F{(i)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(i)}})}}} < {1 - \frac{p}{2}}}{{Object\_ count}\left( {i,t} \right)}}*{\sum\limits_{{{{{{{i \in s}\&}\quad{R{({F{(i)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(i)}})}}} < {1 - \frac{p}{2}}}{F\left( {i,t} \right)}}}$

Where R(F(i)) is the rank percentage of the feature F for FOV i.

The sample summary features for a FOV feature F also include the robustdispersion features such as the mean absolute deviation, interquartilerange and standard errors.

The current invention includes generalized trimming. The generalizedsample trim mean for feature F using both the p percentage of F featureand q percentage of Q featture can be defined as follows:${{Sample\_ GTrim}{\_ Mean}^{pq}{\_ F}\text{:}\quad{\mu_{Sample}^{{pq}\quad F}(t)}} = {\frac{1}{\sum\limits_{{{{{{{{{{{{{i \in s}\&}\quad{R{({F{(i)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(i)}})}}} > {1 - \frac{p}{2}}}\&}\quad{R{({Q{(i)}})}}} > \frac{q}{2}}\&}\quad{R{({Q{(i)}})}}} < {1 - \frac{q}{2}}}{{Object\_ count}\left( {i,t} \right)}}*{\sum\limits_{{{{{{{{{{{{{i \in s}\&}\quad{R{({F{(i)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(i)}})}}} < {1 - \frac{p}{2}}}\&}{R{({Q{(i)}})}}} > \frac{q}{2}}\&}\quad{R{({Q{(i)}})}}} < {1 - \frac{q}{2}}}{F\left( {i,t} \right)}}}$IV.2 Sample Regulated Feature Extraction

The processing flow of the sample regulated feature extraction method isshown in FIG. 14. The sample regulated feature extraction method inputsthe FOV feature results 1300, 1302 from the sample and performs controlFOV selection step 1408 that selects the control FOV 1400 from the FOVfeatures of the sample. The control FOVs 1400 are used by a sampleregulation feature extraction step 1414 to extract sample regulationfeatures 1402. The sample regulation features 1402 are used by an FOVfeature sample regulation step 1410 to generate sample regulated FOVfeatures 1404. The sample regulated FOV features 1404 are used by aregulated sample summary feature extraction step 1412 to generateregulated sample summary features 1406.

IV.2.1 Control FOV Selection

In one embodiment of the invention, the control FOV could be thespecially prepared standard FOVs. The control FOV are selected based onthe assay design. In another embodiment of the invention, the controlFOVs are the reference FOVs extracted from the FOV population. In thecase, the control FOVs are selected based on the sample FOV featuredistribution. For example, the control FOVs could be the FOVs having thearea and mean intensity within the middle 50% of the distribution withinthe sample.

IV.2.2 Sample Regulation Feature Extraction

Sample regulation features can be calculated from the FOV featureresults of the control FOVs for the sample. In one embodiment of theinvention, the sample summary features are extracted for the sampleregulation features. The sample summary features that are suitable forthe sample regulation features include center (mean, median, trim mean,generalized trim mean, etc.) and dispersion (variance, mean absolutedeviation, range, etc.) for FOV features.

IV.2.3 FOV Feature Sample Regulation

The FOV feature sample regulation step regulated the extracted FOVfeatures to create sample regulated FOV features for each of the FOVsbeing considered. It inputs a FOV feature and sample regulation featuresand applied sample regulation formula to the FOV feature. This resultsin sample regulated FOV feature. In one embodiment of the invention, thesample regulation formula is as follows:$F_{Sample\_ r} = \frac{F - {\theta\quad R_{Sample}^{1}}}{\gamma + {\left( {1 - \gamma} \right)\quad R_{Sample}^{2}}}$

Where F is the input FOV feature; θ is a normalization factor; R¹_(Sample) is the first sample regulation feature such as the centerfeature; γ is a weighting factor between 0 and 1; and R² _(Sample) isthe second sample regulation feature such as the dispersion feature.When γ=1, the sample regulation includes only the offset of the featureby the first sample regulation feature. When θ=0 and γ<1, the sampleregulation includes only the gain adjustment of the feature by thesecond sample regulation feature. When θ≠0 and γ<1, the sampleregulation includes both the offset by the first sample regulationfeature and gain adjustment by the second sample regulation feature.

The FOV feature sample regulation allows the removal of the samplespecific bias or background noise and variations. The removal of samplespecific variations would enhance the repeatability and robustness ofthe sample level analysis.

IV.2.4 Regulated Sample Summary Feature Extraction

The regulated sample summary feature extraction inputs the sampleregulated FOV features from a plurality of the FOVs and generates theregulated sample summary features. The same procedure as the robustsample summary feature extraction as described in section IV.1 could beapplied to the sample regulated FOV features to generate the regulatedsample summary features.

V. Assay Level Robust Analysis

The assay level analysis step processes the sample results from thesamples contained in the same assay to create an assay result output. Anassay often contains a plurality of samples. In one embodiment of theinvention, an assay corresponds to a 384 well plate in a well platebased high throughput/high content screening assays. In anotherembodiment of the invention, an assay corresponds to a set of slides ina multiple slide based assays. The assay result could contain one ormore of the attributes such as assay object counts, assay object featurepopulation statistics, assay object relational features, assay samplefeature population statistics, etc.

The assay level analysis inputs a plurality of the sample featureresults 1500, 1502 from the samples belonging to the same assay. Theprior art assay level analysis method extracts assay summary featuresthrough simple population statistics from the sample features. The assaylevel robust analysis method performs robust assay summary featureextraction 1508 using the plurarity of sample feature results 1500, 1502as shown in FIG. 15. This results in robust assay summary features 1504.In addition, an assay regulated feature extraction stage 1510 could beapplied to extract assay regulated features from the plurarity of samplefeature results 1500, 1502. The processing flow of the assay levelrobust analysis method is shown in FIG. 15.

V.1 Robust Assay Summary Feature Extraction

The basic assay summary features are simple population statistics fromthe sample features. Example features include object counts for eachobject type that can be calculated as follows:${{Object\_ count}^{a}(t)\text{:}\quad N_{t}^{a}} = {\sum\limits_{s \in a}{{Object\_ count}\left( {s,t} \right)}}$

Where t is the object type and Object_count(s,t) is the type t objectcount of sample s belonging to assay a.

Other example features are the assay summary statistics of samplefeatures. The sample features are described in section IV such as samplesummary features and sample regulated features, etc. They consist ofbasic features and robust features. In one embodiment of the invention,the assay summary statistics include central tendency and dispersionstatistics. These include both non-robust and robust statistics.

In one embodiment of the invention, the assay summary features for asample feature F include the basic central tendency feature (mean) andbasic spread feature (variance) of assay data such as: $\begin{matrix}{{{Assay\_ Mean}{\_ F}\text{:}\quad{\mu_{Assay}^{F}(t)}} = {\frac{1}{N_{t}^{a}}\quad{\sum\limits_{s \in a}{F\left( {s,t} \right)}}}} \\{{{Assay\_ Variance}{\_ F}\text{:}\quad{\sigma_{Assay}^{2\quad F}(t)}} = {\frac{1}{N_{t}^{a} - 1}{\sum\limits_{s \in a}\left( {{F\left( {s,t} \right)} - {\mu_{Assay}^{F}(t)}} \right)^{2}}}}\end{matrix}$

Those skilled in the art should recognize other summary statistics suchas skewness (third order moment) and Kurtosis (fourth order moment) etc.could be included. Also, other features derived from the estimation ofparameters of different models could be used.

In another embodiment of the invention, the assay summary features for asample feature F include the robust central tendency features such asmedian or trim means that are the averages of the distribution aftercertain percentages (for example, 1%, 5%, 10%, etc.) of data have beentrimmed from the tails of the distribution. Furthermore, Tukey's trimmean, defined as the weighted average of the first, second, and thirdquartile points in a distribution, with weights ¼, ½, and ¼ can also beused. The trim mean at the p percentage can be defined as follows:${{Assay\_ Trim}{\_ Mean}^{p}{\_ F}\text{:}\quad{\mu_{Assay}^{p\quad F}(t)}} = {\frac{1}{\sum\limits_{{{{{{{s \in a}\&}\quad{R{({F{(s)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(s)}})}}} < {1 - \frac{p}{2}}}{{Object\_ count}\left( {s,t} \right)}}*{\sum\limits_{{{{{{{s \in a}\&}\quad{R{({F{(s)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(s)}})}}} < {1 - \frac{p}{2}}}{F\left( {s,t} \right)}}}$

Where R(F(s)) is the rank percentage of the feature F for sample s.

The assay summary features for a sample feature F also include therobust dispersion features such as the mean absolute deviation,interquartile range and standard errors.

The current invention includes generalized trimming. The generalizedsample trim mean for feature F using both the p percentage of F featureand q percentage of Q featture can be defined as follows:${{Assay\_ GTrim}{\_ Mean}^{pq}{\_ F}\text{:}\quad{\mu_{Assay}^{{pq}\quad F}(t)}} = {\frac{1}{\sum\limits_{{{{{{{{{{{{{s \in a}\&}\quad{R{({F{(s)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(s)}})}}} < {1 - \frac{p}{2}}}\&}\quad{R{({Q{(s)}})}}} > \frac{q}{2}}\&}\quad{R{({Q{(s)}})}}} < {1 - \frac{q}{2}}}{{Object\_ count}\left( {s,t} \right)}}*{\sum\limits_{{{{{{{{{{{{{s \in a}\&}\quad{R{({F{(s)}})}}} > \frac{p}{2}}\&}\quad{R{({F{(s)}})}}} < {1 - \frac{p}{2}}}\&}{R{({Q{(s)}})}}} > \frac{q}{2}}\&}\quad{R{({Q{(s)}})}}} < {1 - \frac{q}{2}}}{F\left( {s,t} \right)}}}$V.2 Assay Regulated Feature Extraction

The processing flow of the assay regulated feature extraction method isshown in FIG. 16. The assay regulated feature extraction method inputsthe sample feature results 1500, 1502 from the assay and performscontrol sample selection step 1608 that selects the control samples 1600from the sample features of the assay. The control samples 1600 are usedby an assay regulation feature extraction step 1614 to extract assayregulation features 1602. The assay regulation features 1602 are used bya sample feature assay regulation step 1610 to generate assay regulatedsample features 1604. The assay regulated sample features 1604 are usedby a regulated assay summary feature extraction step 1612 to generateregulated assay summary features 1606.

V.2.1 Control Sample Selection

In one embodiment of the invention, the control sample could be thespecially prepared standard samples. The control samples are selectedbased on the assay design. In another embodiment of the invention, thecontrol samples are the reference samples extracted from the samplepopulation. In the case, the control samples are selected based on theassay sample feature distribution. For example, the control samplescould be the samples having the area and mean intensity within themiddle 50% of the distribution within the assay.

V.2.2 Assay Regulation Feature Extraction

Assay regulation features can be calculated from the sample featureresults of the control samples for the assay. In one embodiment of theinvention, the assay summary features are extracted for the assayregulation features. The assay summary features that are suitable forthe assay regulation features include center (mean, median, trim mean,generalized trim mean, etc.) and dispersion (variance, mean absolutedeviation, range, etc.) for sample features.

V.2.3 Sample Feature Assay Regulation

The sample feature assay regulation step regulated the extracted samplefeatures to create assay regulated sample features for each of thesamples being considered. It inputs a sample feature and assayregulation features and applied assay regulation formula to the samplefeature. This results in assay regulated sample feature. In oneembodiment of the invention, the assay regulation formula is as follows:$F_{Assay\_ r} = \frac{F - {\theta\quad R_{Assay}^{1}}}{\gamma + {\left( {1 - \gamma} \right)\quad R_{Assay}^{2}}}$

Where F is the input sample feature; θ is a normalization factor; R¹_(Assay) is the first assay regulation feature such as the centerfeature; γ is a weighting factor between 0 and 1; and R² _(Assay) is thesecond assay regulation feature such as the dispersion feature.

When γ=1, the assay regulation includes only the offset of the featureby the first assay regulation feature. When θ=0 and γ<1, the assayregulation includes only the gain adjustment of the feature by thesecond assay regulation feature. When θ≠0 and γ<1, the assay regulationincludes both the offset by the first assay regulation feature and gainadjustment by the second assay regulation feature.

The sample feature assay regulation allows the removal of the assayspecific bias or background noise and variations. The removal of assayspecific variations would enhance the repeatability and robustness ofthe assay level analysis.

V.2.4 Regulated Assay Summary Feature Extraction

The regulated assay summary feature extraction inputs the assayregulated sample features from a plurality of the samples and generatesthe regulated assay summary features. The same procedure as the robustassay summary feature extraction as described in section V.1 could beapplied to the assay regulated sample features to generate the regulatedassay summary features.

The invention has been described herein in considerable detail in orderto comply with the Patent Statutes and to provide those skilled in theart with the information needed to apply the novel principles and toconstruct and use such specialized components as are required. However,it is to be understood that the inventions can be carried out byspecifically different equipment and devices, and that variousmodifications, both as to the equipment details and operatingprocedures, can be accomplished without departing from the scope of theinvention itself.

1. An object segmentation confidence mapping method for analysis ofbiological activity comprising the steps of: a) Input an image; b)Perform segmentation decision using the input image having segmentationdecision result output; c) Input a threshold; d) Perform differenceoperation using the segmentation decision result and the thresholdhaving difference result output; e) Perform confidence mapping using thedifference result having segmentation confidence output.
 2. The methodof claim 1 wherein the confidence mapping is determined using at leastone training image.
 3. A robust object segmentation method for analysisof biological activity comprising the steps of: a) Input an image; b)Perform segmentation confidence mapping using the input image havingsegmentation confidence map output; c) Perform thresholding using theobject segmentation confidence map having a high confidence object maskoutput.
 4. The method of claim 3 wherein the thresholding method has alow confidence object mask output.
 5. The method of claim 3 furtherperforms confidence based measurements.
 6. The method of claim 3 whereinthe object segmentation confidence mapping method further comprises thesteps of: a) Perform segmentation decision using the input image havingsegmentation decision result output; b) Input a threshold; c) Performdifference operation using the segmentation decision result and thethreshold having difference result output; d) Perform confidence mappingusing the difference result having segmentation confidence output.
 7. Anobject level robust analysis method for biological activity comprisingthe steps of: a) Input an image; b) Perform object segmentation usingthe input image having object segmentation result output; c) Performrobust object feature measurement using the object segmentation resultand input image having robust object feature result output.
 8. Themethod of claim 7 wherein the robust object feature measurement measuresrobust feature selected from the set consisting of a) Robust estimate ofcentral tendency of the data, b) Robust estimate of dispersion of thedata, c) Robust estimate of general Features.
 9. The method of claim 8wherein the robust object feature measurements confidence basedmeasurements.
 10. A FOV level robust analysis method for biologicalactivity comprising the steps of: a) Input a plurality of object featureresults; b) Perform robust FOV summary feature extraction using theplurality of object feature results having robust FOV summary featuresoutput; c) Perform FOV regulated feature extraction using the pluralityof object feature results having FOV regulated features output.
 11. Themethod of claim 10 further performs object relational feature extractionusing the plurality of object feature results having robust FOVrelational features output.
 12. The method of claim 10 wherein therobust FOV summary feature extraction measures robust FOV summaryfeature selected from the set consisting of a) Robust estimate forsummary feature of central tendency of the FOV data, b) Robust estimatefor summary feature of dispersion of the FOV data, c) Robust estimatefor summary general features of the FOV data.
 13. The method of claim 10wherein the FOV regulated feature extraction further comprises the stepsof: a) Perform control object selection using the plurality of objectfeature results having control objects output; b) Perform FOV regulationfeature extraction using the control objects having FOV regulationfeatures output; c) Perform object feature FOV regulation using theplurality of object feature results and the FOV regulation featureshaving FOV regulated object features output.
 14. The method of claim 10further performs regulated FOV summary feature extraction using theplurality of object feature results having robust FOV relationalfeatures output.
 15. A FOV regulated feature extraction method forbiological activity comprising the steps of: a) Input a plurality ofobject feature results; b) Perform control object selection using theplurality of object feature results having control objects output; c)Perform FOV regulation feature extraction using the control objectshaving FOV regulation features output; d) Perform object feature FOVregulation using the plurality of object feature results and the FOVregulation features having FOV regulated object features output.
 16. Themethod of claim 15 further performs regulated FOV summary featureextraction using the plurality of FOV regulated object features havingregulated FOV summary features output.
 17. The method of claim 15 wherein the FOV regulation feature extraction calculates using the followingformula:$F_{FOV\_ r} = \frac{F - {\theta\quad R_{FOV}^{1}}}{\gamma + {\left( {1 - \gamma} \right)R_{FOV}^{2}}}$18. A sample level robust analysis method for biological activitycomprising the steps of: a) Input a plurality of FOV feature results; b)Perform robust sample summary feature extraction using the plurality ofFOV feature results having robust sample summary features output; c)Perform sample regulated feature extraction using the plurality of FOVfeature results having sample regulated features output.
 19. The methodof claim 18 wherein the robust sample summary feature extractionmeasures robust sample summary feature selected from the set consistingof a) Robust estimate for summary feature of central tendency of thesample data, b) Robust estimate for summary feature of dispersion of thesample data, c) Robust estimate for summary general features of thesample data.
 20. The method of claim 18 wherein the sample regulatedfeature extraction further comprises the steps of: a) Perform controlFOV selection using the plurality of FOV feature results having controlFOV output; b) Perform sample regulation feature extraction using thecontrol FOV having sample regulation feature output; c) Perform FOVfeature sample regulation using the plurality of FOV feature results andthe sample regulation feature having sample regulated FOV featuresoutput.
 21. The method of claim 18 further performs regulated samplesummary feature extraction using the sample regulated FOV featureresults having regulated sample summary features output.
 22. An assaylevel robust analysis method for biological activity comprising thesteps of: a) Input a plurality of sample feature results; b) Performrobust assay summary feature extraction using the plurality of samplefeature results having robust assay summary features output; c) Performassay regulated feature extraction using the plurality of sample featureresults having assay regulated features output.
 23. The method of claim22 wherein the robust assay summary feature extraction measures robustassay summary feature selected from the set consisting of a) Robustestimate for summary feature of central tendency of the assay data, b)Robust estimate for summary feature of dispersion of the assay data, c)Robust estimate for summary general features of the assay data.
 24. Themethod of claim 22 wherein the assay regulated feature extractionfurther comprises the steps of: a) Perform control sample selectionusing the plurality of sample feature results having control sampleoutput; b) Perform assay regulation feature extraction using the controlsample having assay regulation feature output; c) Perform sample featureassay regulation using the plurality of sample feature results and theassay regulation feature having assay regulated sample features output.25. The method of claim 22 further performs regulated assay summaryfeature extraction using the assay regulated sample feature resultshaving regulated assay summary features output.